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r\l • Abstract. In this paper, we propose an approach to reduce the optimal 

controller synthesis problem of hybrid systems to quantifier elimination; 
furthermore, we also show how to combine quantifier elimination with nu- 

^H ■ merical computation in order to make it more scalable but at the same 

^^ ' time, keep arising errors due to discretization manageable and within 

jy! , bounds. A major advantage of our approach is not only that it avoids 

^ ' errors due to numerical computation, but it also gives a better optimal 

controller. In order to illustrate our approach, we use the real indus- 
trial example of an oil pump provided by the German company HYDAC 
within the European project Quasimodo as a case study throughout this 
paper, and show that our method improves (up to 7.5%) the results 

fvq ■ reported in [3] based on game theory and model checking. 
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1 Introduction 



►v> ' Hybrid systems such as physical devices controlled by computer software, are 

5^ , systems that exhibit both continuous and discrete behaviors. Controller synthe- 

_Cy_' sis for hybrid systems is an important area of research in both academia and 

industry. A synthesis problem focuses on designing a controller that ensures 
the given system will satisfy a safety requirement, a liveness requirement (e.g. 
reachability to a given set of states), or meet an optimality criterion, or a desired 
combination of these requirements. 

Numerous work have been done on controller synthesis for safety and/or 
reachability requirements. For example, in [1I27J . a general framework for syn- 
thesizing controllers based on hybrid automata to meet a given safety require- 
ment was proposed, which relies on backward reachable set computation and 
fixed point iteration; while in |24| . a symbolic approach based on templates and 
constraint solving to the same problem was proposed, and in [25], the symbolic 
approach is extended to meet both safety and reachability requirements. 



However, the optimal controller synthesis problem is more involved, also quite 
important in the design of hybrid systems. In the literature, few work has been 
done on the problem. Larsen et al proposed an approach based on energy au- 
tomata and model-checking [3] , while Jha, Seshia and Tiwari gave a solution to 
the problem using unconstrained numerical optimization and machine learning 
|14| . However, in [3], allowing control only to be exercised at discrete points 
in time certainly limits the opportunity of synthesizing the optimal controller 
(though one can get arbitrarily close). Moreover, discretizing could cause an 
incorrect controller to be synthesized — which therefore requires a posterior 
analysis (e.g. in [3], PHAVER [^ is used for the purpose). The approach of [T4] 
suffers from imprecision caused by numerical computation, and cannot synthe- 
size a really optimal controller sometimes because the machine learning technique 
cannot guarantee its completeness. 

In this paper, we propose a "hybrid" approach for synthesizing optimal con- 
trollers of hybrid systems subject to safety requirements. The basic idea is as 
follows. Firstly, we reduce optimal controller synthesis subject to safety require- 
ments to quantifier elimination (QE for short). Secondly, in order to make our 
approach scalable, we discuss how to combine QE with numerical computation, 
but at the same time, keep arising errors due to discretization manageable and 
within bounds. A major advantage of our approach is not only that it avoids 
errors due to numerical computation, but it also gives a better optimal controller. 

Application of QE in controller synthesis of hybrid systems is not new. The 
tool HyTcch was the first symbolic model checker that can do parametric analysis 
[12] for linear hybrid automata, but for the oil pump example it will abort 
soon due to arithmetic overflow errors. Recently, verification and synthesis of 
switched dynamical systems using QE were discussed in [23] , where the authors 
gave principles and heuristics for combining different tools, to solve QE problems 
that are out of the scope of each component tool. 

Our encoding of a MIN-MAX-MIN optimization problem into a QE problem 
is inspired by the idea in [7]: minimizing an objective function f{xi, X2-, ■ ■ ■ ,Xn) 
can be solved by introducing an additional constraint z > f{xi, X2, . . . , x„) and 
eliminating variables xi, 2:2, . . . , a;„, where z is a newly introduced variable. Sim- 
ilar ideas can also be found in [4]. 

The computation of optimal control strategies in this paper is typically a 
parametric optimization problem, a topic researched extensively in both oper- 
ation research and control communities. Symbolic methods have advantages in 
addressing parametric optimization problems |28I8I15J . However, we do not find 
any algorithm suitable for solving a parametric quadratic optimization problem 
over constraint with complex Boolean structure and hundreds of (or thousands 
of) atomic formulas as in this paper. 

It was shown in 12^ that for certain parametric quadratic optimization prob- 
lems, the closed form solution exists: the optimizer is a piecewise affine function 
in the parameters, and the optimal value is a piecewise quadratic function in the 
parameters. Our experiment results confirm this. 



In order to illustrate our approach, we use the oil pump industrial example 
provided by the German company HYDAC within the European project Quasi- 
modo as a case study throughout this paper, and show that our method results 
in a better optimal controller (up to 7.5% improvement) than those reported 
in [3] based on game theory and model checking. Moreover, we prove that the 
theoretically optimal controller of the oil pump example can be synthesized and 
its correctness is also guaranteed with our approach. 

Paper Organization: In Section 2 we propose a general framework for opti- 
mal controller synthesis of hybrid systems based on quantifier elimination and 
numerical computation. We focus on the oil pump case study in Section 3-6: a 
description of the oil pump control problem is given in Section 3, modeling of 
the system and safety requirements is shown in Section 4, a "hybrid" approach 
for performing optimization is presented in Section 5, and further improvement 
through a modification in the model is discussed in Section 6. We finally conclude 
this paper by Section 7. 

2 The Overall Approach 

In this section we propose an approach that reduces optimal controller synthesis 
of hybrid systems subject to safety requirements to QE. Such reduction is based 
on reachable set computation or approximation of hybrid systems and symbolic 
optimization. We also discuss how numerical computation can be incorporated 
into our approach to make it more scalable. 

Generally, a hybrid system consists of a set of continuous state variables x 
(ranging over M") and a set of discrete operating modes Q, with each of which 
a continuous dynamics is associated specifying the behavior of x at each mode; 
discrete jumps between different modes may happen if some transition conditions 
are satisfied by x. 

The optimal controller synthesis problem studied in this paper can be stated 
as follows. Suppose we are given an under-specified hybrid system T-L, in which 
the transition conditions are not determined but parameterized by u, a vector of 
control parameters. Our task is to determine values of u such that H can make 
discrete jumps at desired points, thus guaranteeing that 

1) a safety requirement iS is satisfied, that is, x stays in a designated safe region 
at any time point; and 

2) an optimization goal Q, possibly 

min(;(u), maxmin (7(u) , or minmaxmin(7(u) ,lj 

U U2 Ui Us U2 Ui 

where .g(u) is an objective function in parameters u, is achieved. 

Then our approach for solving the synthesis problem can be described as the 
following steps. 



^ We assume that u is chosen from a compact set, and elements of u are divided into 
groups ui, U2, U3, . . . according to their roles in Q. 



Step 1. Derive constraint D(u) on u from safety requirements of the system. 

If the reachable set R (parameterized by u) of H can be exactly computed 
(e.g. for very simple linear hybrid automata), then we just require that R should 
be contained in the safe region. Otherwise we have to approximate R (with 
sufficient precision) by automatically generating inductive invariants of T-i (e.g. 
for general linear or nonlinear hybrid systems). The notion of inductive invariant 
is crucial in safety verification of hybrid systems |10I21[ . and constraint-based 
approaches have been proposed for automatic generation of inductive invariants 
|2:j|l0l20ll6j . 

Step 2. Encode the optimization problem Q over constraint Din) into a quan- 
tified first- order formula Qu.iy9(u, z), where z is a fresh variable. 
Our encoding is based on the following proposition. 

Proposition 1. S'uppo.se gi(ui), g2(ui, U2), .g3(ui, U2, U3) are polynomials, and 
Z?i(ui), Z?2(ui,U2), D3(ui, U2, U3) are nonempty compact semi- algebraic seio 
Then there exist ci, C2, C3 G R s.t. 

3ui.{DiAgi <z) <=> z>ci, (1) 

VU2.(3ui.i:»2 > 3UI.{D2 hg2<z)) ^=^ Z > C2 , (2) 

3u3.((3uiU2.L'3) A Vu2.(3ui.A3 ^3ui.(L'3A53<2))) ^=^ z > C3 , (3) 
where >e {>,>}, an(ici,C2,C3 satisfy 

ci = mingi(ui) o\ct: Di{vli) , (4) 

ui 

C2 = supmin52(ui,U2) overZ?2(ui, U2) , (5) 

U2 Ui 

C3 = infsupmin53(ui,U2,U3) over 153 (ui, U2, U3) . (6) 

U3 U2 Ui 

Proof. Given assumptions in Proposition[l] the following facts are easy to check: 

(fl) 3ui.L)2(ui, U2) is a compact set over U2; 

(f2) for any vL^ satisfying 3ui._D2(ui, U2), the instantiation of D2 by Uj, i.e. 

-D2(ui,U2) is a compact set over Ui; 
(f3) results similar to (fl) and (f2) can be established for _D3(ui, U2, U3) . 

First we show the existence of ci , C2 , C3 in @, ([5]) and (O . 

Proof of pp.- The existence of ci is based on the Extreme Value Theorem: a 
real- valued continuous function has a minimum and a maximum on a compact 
set. 

Proof of (0); Let 

C2 = max (72(ui,U2) over Z)2(ui,U2). 

Ul,U2 



^ A semi-algebraic set is defined by Boolean combinations of polynomial equations 
and inequalities. 



Then for any Cj satisfying Elui.Z?i 

ming2(ui,U2) over i:)2(ui,U2) 

ui 

exists and 

niin.g2(ui,U2) < cj. 

ui 

Therefore the supremum of minm .92 (ui, U2) over D2, i.e. 02, exists. 
Proof of p.- Let 

C3 = min g3(ui,U2,U3) over D3 . 

Ui,U2,U3 

Then 

supniin(73(ui,U2,U3) over D3 

U2 Ul 

is lower bounded by C3. Thus C3 exists. 

Next we wiU prove (Ul - (121). For brevity, in the sequel we use (•)' and (•)'' to 
denote the left and right hand side sub-formulas in the equivalence relations ([1]) 
-©. 

Proof of (pp.' "=>" Suppose z satisfies ([1])' but z < ci. Then there exists 
u^; G Di s.t. 

ci> z> gi{ul), 

which contradicts @; "<^" Suppose z satisfies ([T])''. By (g]) we have ci = 51 (u^) 
for some u* G I?i. Thus 

z>ci = gi{ul), 

so z satisfies ([T])'. 

Proof of (0).' "=^" Suppose z satisfies ^K Then for all Uj in 3ui.D2 we have 

3ui.(D2(ui,U2) A,92(U1,U2) < z) . 

By dU it follows that 

z > ming2(ui, U2) over _D2(ui,U2) 

Ul 

for all U2 in 3ui.D2, so by ([5]) z > C2 . "<^" Suppose z satisfies (O*". Then by 
([5]) we have for all U2 in Elui._D2 

z > min52(ui, U2) over _D2(ui,U2). 

Ul 

Again by (P) we get 

3ui.(i:)2(ui,U2) A.g2(ui,U2) < z) 

holds for all U2 in 3ui._D2, which means z satisfies ^^ . 

Proof of (OJ; The proof below is based on the fact that if infimum in ^ is 
actually minimum, then Q'' is z > C3; otherwise ([3])'" is 2; > C3. We only give 
the proof for the former case. 



"=^" Suppose z satisfies ([3])'. Then there exists Ug in 3uiU2.£'3 s.t. 

Vu2.(3ui.i:)3(ui,U2,U3) > 3ui.(i:'3(ui,U2,U3) A ^3 (ui , U2 , U3) < z)) . 

By ([2]) we have 

z > supming3(ui,U2,U3) over ^'(ui, U2, U3) . 

Then by © 2; satisfies (jH])''. "^" Suppose z satisfies ([3])''. Then by ([5]) we assert 
that there exists Ug in 3U1U2.-D3 s.t. 

z > supming3(ui,U2,U3) over I?(ui, U2, U3) . 

U2 Ul 

Again by ([2]) it follows that z satisfies 

Vu2.(3ui.i:)3(ui,U2,U3) > 3ui.(i:)3(ui,U2,U3) A ^3 (Ui , U2 , U3) < z)) . 

Thus z satisfies ([3])'. 

If infimum in ([5]) is not minimum, an analogous proof can be given. D 



Step 3. Eliminate quantifiers in Qu.iy9(u, z) and from the result we can retrieve 
the optimal value of G and the corresponding optimal controller u. 

By Proposition [H the optimal value of a MIN, MAX-MIN or MIN-MAX-MIN 
problem can be obtained by applying QE to the left hand side (LHS) formulas in 
([T|)-([3]) respectively. Although QE for the first-order theory of real closed fields 
is a complete decision procedure [5B], due to the inherent doubly exponential 
complexity [S], we cannot expect to compute an optimal value, say C3, by directly 
applying QE to a big formula with many alternations of quantifiers, like LHS of 
([3]). Therefore it is necessary to devise our own mechanisms for performing QE 
more efficiently. 

Note that in ([3|), any instantiation of the outmost quantified variables U3 
would result in a simpler formula, whose quantifier-free equivalence gives an up- 
per bound of C3. If in some way we know the bounds of U3, i.e. h < u^ < Ui, for 
1 < I < dim(u3), then by discretizing U3 over all [li,Ui] with certain granularity 
A, and using the set of discretized values to instantiate the outmost existential 
quantifiers of ([31), we can get a finite set of simplified formulas, each of which 
produces an upper approximation of C3. Finally, through an exhaustive search in 
this set we can select such an approximation that is closest to C3 . Finer granular- 
ity yields better approximation of the optimal value, so one can seek for a good 
balance between timing and optimality by tuning the granularity A. Further- 
more, the above computation is well suited for parallelization to make full use 
of available computing resources, because the intervals [Z^, Ui] and corresponding 
instantiations can be divided into subgroups and allocated to different processes. 



3 Description of the Oil Pump Control Problem 

The oil pump example [3] was a real industrial case provided by the German 
company HYDAC ELECTRONICS GMBH, and studied at length within the Euro- 
pean research project Quasimodo. The whole system, depicted by Fig.[Tl consists 
of a machine, an accumulator, a reservoir and a pump. The machine consumes 
oil periodically out of the accumulator with a duration of 20 s (second) for one 
consumption cycle. The profile of consumption rate is shown in Fig. [2l The pump 
adds oil from the reservoir into the accumulator with power 2.2 1/ s (liter/second). 
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Fig. 1. The oil pump system. (This pic- Fig. 2. Consumption rate of the machine in one 
ture is based on [3].) cycle. 



Control objectives for this system are: by switching on/ofF the pump at cer- 
tain time points 

< fi < t2 < • ■ • < i„ < t„+i < • • • , (7) 

ensure that 

• Rs {safety): the system can run arbitrarily long while maintaining v{t) within 
[Knin, Knax] for any time point i, where v{t) denotes the oil volume in the 
accumulator at time t, Vinin = 4.9/ (liter) and Vniax = 25.1 1 ; 

and considering the energy cost and wear of the system, a second objective: 



• Ro (optimality): minimize the average accumulated oil volume in the limit, 
i.e. minimize 



lim 



1 



T^oo T ./t=o 



v(t)dt 



Both objectives should be achieved under two additional constraints: 

• Rpi {pump latency): there must be a latency of at least 2 s between any two 
consecutive operations of the pump; and 

• Ri- (robustness): uncertainty of the system should be taken into account: 

- fluctuation of consumption rate (if it is not 0), up to / = 0.1 l/s ; 

- imprecision in the measurement of oil volume, up to e = 0.06/ ; 



- imprecision in the measurement of time, up to S = 0.015s|j 

In [3], the authors used timed game automata to model the above system, 
and apphed the tool UPPAAL-TIGA to synthesize near-optimal controllers. Due 
to discretization made in the timed-game model, an incorrect controller might 
be synthesized. Therefore the correctness and robustness of the synthesized con- 
trollers are checked using the tool PHAVER. Through simulations in SIMULINK, 
it was shown that the controllers synthesized by UPPAAL-TIGA provides big im- 
provement (about 40%) over the Bang-Bang Controller and Smart Controller 
that are currently used at the HYDAC company. We will show how further im- 
provement can be achieved using our approach. 



4 Deriving Constraints from Safety Requirements 

Following [5], the determination of control points ([7]) can be localized by ex- 
ploiting the periodicity of oil consumption. That is, decisions on when to switch 
on/off the pump in one cycle can be made locally by measuring the initial oil 
volume vq at the beginning of each cycle. Accordingly, the safety requirement 
Rs in Section[3]can be reformulated as: find an interval [L, U] C [I/min, Knax] s.t. 

• Riu {constraint for L,U): for all vq S [^jt^], there is a finite sequence of 
time points t = (ti, ^2, ■ ■ • , ^n) U where < ti < ^2 < ■ • ■ < in < 20 satisfy 
Rpi, for turning on/off the pump so that the resulting v{t) with v{0) = vq 
satisfies 

• Rj (inductiveness): v(20) G [i,C/]; and 
. Ris {local safety): v{t) G [Knm,Knax] for ah t G [0,20] 
under constraint Rj.. 

Definition 1 (Local Controller). The above t corresponding to vq is called a 
local controller; the interval [L, U] is called a stable interval. 

Basically, Riu says that there is a stable interval [L, U] and a corresponding 
family of local control strategies which can be repeated for arbitrarily many 
cycles and guarantee safety in each cycle. 



Modeling Oil Consumption. Let Voutit) with Vout{0) = denote the amount 
of oil consumed by time t in one cycle, and modify the consumption rate in 
Fig. [2] by / in (Rr). Then by simply integrating the lower and upper bounds of 



^ In [3], 6 is assumed to be 0.01. Here we include an extra rounding error of 0.005 due 
to floating point calculations in the implementation of our control strategy. 

* The choice of n will be made later (in this paper n can be 0, 2, 4, 6), but larger n's 
obviously will have the potential of allowing improved controllers. 



the consumption rate over the time interval [0, 20] we can get 

(0<t<2 — > Vout=0) 

A(2<t<4 > l.l{t-2)<Vout<l-3{t-2)) 

A(4<t<8 > 2.2<Vo„t<2.6) 

A(8<t<10 — > 2.2+l.l(t-8)<yout<2.6+1.3(t-8)) 
Ci = A(10<t<12 — > 4.4+2. 4(t-10)<Vo„t<5. 2+2. 6(t-10)) 

A(12<t<14 > 9.2<K,ut<10.4) 

A(14<t<16 — > 9.2+1.6(«-14)<Vo„4<10.4+1.8(t-14)) 
A(16<t<18 — > 12.4+0.4(t-16)<V'„uf<14+0.6(t-16)) 
A(18<t<20 — > 13.2<V'„ut<15.2) 

Actually, if the machine consuming oil is regarded as a hybrid system Ti with 
state variable Vout and continuous dynamics subject to box constraints, then Ci 
is the exact reachable set of Ti from initial point Vout = within 20 time units. 
Therefore wc do not need to approximate the reachable set of T-i by generating 
inductive invariants. This is also the case with the following pump system. How- 
ever, if the consumption profile is more complicated, say piecewisc polynomial, 
then approximations are indeed necessary. 

Modeling Pump. In P] it is assumed that the number of activations of pump 
in one cycle is at most 2. We will adopt this assumption at first and increase this 
number later on. With this assumption, there will be at most four time points 
to switch the pump on/off in one cycle, denoted by < ti < ^2 < ^3 < ^4 < 20. 
If the pump is started only once or zero times, then we just set ^3=^4 = 20 
or ti — t2 = t^ = ti = 20 respectively. Then the 2-second latency requirement 
(Rpi) can be modeled by 

(ti>2 A t2-ti>2 A t3-t2>2 A t4-t3>2 A t4<20) 
€2= V (ti>2 At2-tl>2 At2<20 At3=20 At4=20) 
V (ti=20 A t2=20 A t3=20 A t4=20) 

Let Vin{t) with T^m(O) = denote the amount of oil introduced into the 
accumulator by time t in one cycle. Then we have 

(0<t<ti — > V,„=0) 

A(ti<t<t2 > Vir,=2.2(t-ti)) 

C3= A(t2<t<i3 — > V,„=2.2{t2~ti)) 

A(t3<t<i4 > Vi„=2.2(t2-ti)+2.2{t-t3)) 

A(t4<t<20 > V'i„=2.2(i2+t4-tl-i3)) 

Encoding Safety Requirements. Denote the oil volume in the accumulator 
at the beginning of one cycle by wq, and the volume at time t by v{t). Then for 
any < i < 20 we have: 



C4 = W = Wo + Vin - Vc 



out 



^ In the sequel when a function 7(f) appears in a formula, the argument t is dropped 
and 7 is taken as a real-valued variable. 



According to (Rr), the measurement of f,; (1 < i < 4) and vq may deviate 
from their actual values, so v{t) will deviate from its predicted value as stated 
in the requirement C4. Nevertheless, we have the following estimation of the 
deviation of v(t). 

Lemma 1. Let v{t) denote the actual oil volume in the accumulator at time t. 
Then for anyO<t< 20, \v{t) - i{t)\ < 8.8S + e < 0.2. 

Proof. By (Rr) and C4, vq will cause an imprecision of e and each ti will cause 
an imprecision of 2.2 (5 . D 

By Lemma [U it is sufficient to rectify the safety bounds in (Ri) and (Ris) by 
an aniomit of 0.2. Let 

C5 = i = 20 — > L + 0.2<v <U -0.2 

Ce = < i < 20 ^ Klin + 0.2 < t- < Knax ^ 0.2 . 

Then (Ri) and (Ris) can be expressed as 

S = yt, V, y„, Vout-{.Ci AC3ACi-^C5A Ce) . 

Deriving Constraints. To find such [L, U] that for every vq G [L, U] there is 
a local control strategy satisfying Rj and Ris, let 

C7= L<vo<U , 

and then Riu can be encoded into 

Cs^yVo.(C7 ^3tit2t3ti.{C2 A5)) . 

We use the tool MjoUnir [TS] to do QE on Cs and the following result is returned: 

Cg = L > 5.1 A [/ < 24.9 A [/ - L > 2.4 . 

Then the relation between L, U, vq and the corresponding local control strategy 
t = (ii, ^2, ^3, ^4) can be obtained by applying QE to 

Cio = C2 A C7 A Cg AS. 

The result given by Mjollnir, when converted to DNF, is a disjunction of 92 

components: 

92 

V{LM,voMMM,ti)= \J D, 

i=l 

(denoted by T) for short), with each Di representing a nonempty closed convex 
polyhedron (see Appendix lA.ip pI 



The fact that each Dt is a nonempty closed set can be checked using QE. 



5 A "Hybrid" Approach for Optimization 

5.1 Encoding of the Optimization Objective 

By Definition [1] the optimal average accumulated oil volume in Ro can be rede- 
fined as 

1 f^° 
• R' : min max min — / v(t)dt . (8) 

[L.U] .0^1L,U] t 20 Jt^o ^ ^ ^ ' 

The intuitive meaning of (R^) is: 

— for each admissible [L,U] and each vq G [L,U], minimize the average accu- 
mulated oil volume in one cycle, i.e. ^ Jt=Qf^{t)d.t, over all admissible local 
controllers t; 

— fix [L, U] and select the worst local minimum by traversing all Vq G [L, U]; 

— then the global minimum is obtained at the interval whose worst local min- 
imum is minimal. 

Definition 2 (Local Optimal Controller). Let Pt = {t | {L,U, vo,t) G V} 

for fixed L, U, vq. Then we call 

20 



1 r^ 

min — / v(t)dt 



the optimal local average accum,ulated oil volume corresponding to L,U,Vo, and 
the optimizer t is called the local optimal controller. 

Let g{vo, ii, <2, ^3, ^4) = ^ /j^g v{t)dt, denoted by g for short. Then it can be 
computed from Ci, C3, C4 without considering fluctuations of consumption rate 
that 

_ 20wo + l.l(i? -tl+tl-tl- 40ti -I- 40^2 - 40^3 -I- 40*4) - 132.2 
^ ~ 20 ■ 

Then by Proposition [1] (R^) can be encoded into 

3L,U.(Cq a Vvo-iCr ^ 3*1*2*3^4.(2? A5 < z))) , (9) 

which is equivalent to z > z* or 2 > z*. where z* equals the value of ([S]). 

5.2 Techniques for Performing QE 

The above deduced (O is a huge formula with nonlinear terms and two alter- 
nations of quantifiers, for which direct QE fails. Therefore we have made our 
efforts to decompose the QE problem into manageable parts. 



Eliminating the Inner Quantifiers. We first eliminate the innermost quan- 
tified variables 3iii2^3i4 by employing the theory of quadratic programming. 

Note that Di in 2? is a closed convex polyhedron for all i and g is a quadratic 
polynomial function, so minimization of g on Di is a quadratic programming 
problem. Then the Karush-Kuhn- Tucker (KKT) [l^ condition 



7kkt 



3/x./:(g,A), (10) 



where C{g, Di) is a linear formula constructed from g and -D^, and /x is a vector 
of new variables, gives a necessary condition for a local minimum of 5 on Di. 

By applying the KKT condition to each Di and eliminating all /x, we can get 
a necessary condition 2?', a disjunction of 580 parts, for the minimum of g on 2): 

580 

j=i 

Furthermore, each Bj has the nice property that for any L, U,vq, a unique tj is 
determined by Bj (see Appendix IA.2[) rl For instance, one of the Bj reads: 

i4 = 20A16t2 + 10L-349 = 0A , . 

i2 - t3 + 2 = A 22f 1 - 16^2 - lOwo + 107 = A ■ • • ■ *- '' 

Since T?' keeps the minimal value point of g on P, the formula obtained by 
replacing V by V in © 

3L, C/.fCg A Vvo-(C7 — > 3tit2hU.{V' Ag< z))) (12) 

is equivalent to dH). Then according to formulas like (fTTj) . 3^1^2*3^4 in (fT2|) can be 
eliminated by the distribution of 3 among disjunctions, followed by instantiations 
of tj in each disjunct. Thus ([9]) can be converted to 

580 

3i, U. (Cg A Vvo-iCr -^ \/ {A, A g, < z))) , (13) 

i=i 

where Aj is a constraint on L,U,vq, and g^ is the instantiation of g using tj 
given by formulas like pT|) . 

Eliminating the Outer Quantifiers. We eliminate the outermost quantifiers 
3L, U in P^ by discretization, as discussed in Section [51 

According to Cg, the interval [5.1,24.9] is discretized with a granularity of 
0.1, which gives a set of 199 elements. Then assignments to L, U from this set 
satisfying Cg are used to instantiate p^ . There are totally 15400 such pairs of 
L, [/, e.g. (5.1, 7.5), (5.1, 7.6) etc, and as many instantiations in the form of 

580 

^v^.{C,^\J{A,Ag,<z)) , (14) 



This has been verified by QE. 



each of which gives an optimal value corresponding to [L,t/]. In practice, we 
start from L ~ 5.1, U = 7.5, and search for the minimal optimal value through 
all the 15400 cases with L or U incremented by 0.1 every iteration. 

Eliminating the Middle Quantifier. Wc finally eliminate the only quantifier 
left in (|14p by a divide- and-conquer strategy. First, wc can show that 

Lemma 2. \J a^^Aj is equivalent to Cj in |_?4[ ). 

Proof. As discussed above, 

580 

3t.(2? A ,9 < z) ^ 3t.(2?' A .9 < z) ^ Y {Aj A g-j < z) . 



Therefore 
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3z3t.{V hg<z)^3z.\l {Aj A Qj < z) 



By eliminating z we have 3t.T> <^ \l j=iAj. According to Cs and Cio, 2? has 
been chosen in such a way that for any wo G [L, U] there is a local controller t. 
Thus 3t.X' <^ C7 when L, U are instantiated. D 



By this lemma if all Aj are pairwise disjoint then ([H]) is equivalent to 
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f\ Vwo-(wo e Aj- ^ (Aj- A Qj < z)) 



(15) 



Since each conjunct in (J15p is a small formula with only two variables vq, z and 
one universal quantifier, it can be dealt with quite efficiently 





Fig. 3. Region partition. 



If the set of AjS are not pairwise disjoint, then we have to partition them 
into disjoint regions and assign a new cost function g^ to each region. The idea 
for performing such partition is simple and is illustrated by Fig. [3l 



Suppose two sets, say Ai,A2, are chosen arbitrarily from the set of ^^s. If 
Ai n A2 = 0, then we do nothing. Otherwise check wether gi < 92 (or 52 < 5i) 
on Ai D A2: if so, assign the smaller one, i.e. gi < z (or 32 < 2;) to Ai Ci A2; 
otherwise we simply assign {gi < z)V (32 < z) to Ai ("1^2. 

If at the same time of partitioning regions we also make a record of the local 
control strategy in each region, i.e. t^, then in the end we can get exactly the 
family of local optimal controllers corresponding to each vq. 

5.3 Results of QE 

Various tools are available for doing QE. In our implementation, the SMT-based 
tool MjoUnir |19I18| is chosen for QE on linear formulas, while REDLOG |6] im- 
plementing virtual substitution jl7) is chosen for formulas with nonlinear terms. 
The computer algebra system REDUCE [TT], of which REDLOG is an integral 
part, allows us to perform some programming tasks, e.g. region partition. Table 
[T]shows the performance of our approach. All experiments are done on a desktop 
running Linux with a 2.66 GHz CPU and 3 GB memory. 

Table 1. Timing of different QE tasks. 



formula 


Cg 


Cio 


Skkt (all 92) 


all the rest 


tool 


MjoUnir 


MjoUnir 


MjoUnir 


Redlog/Reduce 


time 


8m8s 


4ml3s 


31s 


<ls 



Remark In Table [U timing is in minutes (m) and seconds (s); in the last column, 
the time taken to get the first optimal valuci is less than 1 second, whereas all 
15400 iterations will cost more than 10 hours (using a single computing process). 
The final results are as follows: 



— The interval that produces the optimal value is [5.1,7.5]. 

— The local optimal controller for vq £ [5.1,7.5] is 



t^ 



lOwo - 25 



A ts 



10i;o + 1 



A i.s 



lOwo + 153 



A t4 



(16) 



157 

TT 

6.5, then by dH]) the 



13 13 22 

which is illustrated by Picture I in Fig. |4l If v^ 
pump should be switched on at ii = 40/13, off at t2 = 66/13, then on at 
ts = 109/11, and finally off at t^ = 157/11. 

The optimal average accumulated oil volume ^28ct'o ~ '^■^'^ ^^ obtained, im- 
proving by 5% the optimal value 7.95 in [3], which is already a 40% im- 
provement of the controllers from the HYDAC company. The local optimal 
average accumulated oil volume for vq G [5.1, 7.5] under controller (|16|) . i.e. 
y„„(„o) = i3oo„^+.^o^42o„o+a34ai7 ^ -^ illustrated by II of Fig. in 



* For the model with 2 activations, this optimal value is only obtained at the 1st 
iteration. 



From II of Fig. |3]we can have an estimate of the performance of controUer 
(|16p in the long run. Without considering noises, it can be computed from 
P^ that w(20) = 6.3 no matter what v{0) is, implying that the mean value of 
vq equals 6.3. Therefore the mean average accumulated oil volume in the long 
run is Vaa«(6.3) = ^^f = 7.125. In [3]i by simulating the oil pump system 
for a duration of 200s, the mean values 7.44, 11.56 and 13.45 are obtained for 
the UPPAAL-TIGA controller, Smart Controller and Bang-Bang Controller 
respectively. 





Fig. 4. Optimal controllers and average accumulated oil volumes for 2 activations. 



6 Improvement by Increasing Activation Times 



In the controller shown by I of Fig. SI wc noticed that when vq is small and the 
pump is started on for the second time, it stays on for a period longer than 4 
seconds. Based on this observation, we conjecture that if the pump is allowed 
to be activated three times in one cycle, then each time it could stay on for a 
shorter period, and the time it is activated for the third time can be postponed. 
As a result, the accumulated oil volume in one cycle may become less. 

To verify the above conjecture, some modifications must be made on the 
previous model. Firstly, C2 and C3 should be replaced 



C^ = 



(ti>2 At2-tl>2 A t3-t2>2 At4-i3>2 A t5-t4>2 A t6-t5>2 A t6<20) 

V (ti>2 A t2-tl>2 A i3-t2>2 A t4-t3>2 A t4<20 A t5=20 A t6=20) 

V (ti>2 A t2-tl>2 A t2<20 A t3=20 A i4=20 A t5=20 A ts=20) 

V (ti=20 A t2=20 A t3=20 A t4=20 A t5=20 A te=20) 



and 



c- 



{0<t<ti 

A {ti<t<t2 

A {t2<t<ta 

A (t3<t<t4 
A {ti<t<ts 
A(t5<t<t6 

A (t6<t<20 



Vi„=0) 

Vi„=2.2{t~-ti)) 

Vi„=2.2{t2-ti)) 

Vi„=2.2{t2-ti)+2.2(t-t3)) 

Vi„=2.2{t2+t4~tl~t3)) 

Vi„=2.2{t2+t4-ti-t3)+2.2{t-ts)) 
V,„=2.2{t2+t4+te-ti-t3-ti)) 



respectively; secondly, in C5 and Ce the tolerance of noises should be increased 
to 0.3, because due to the increase of times to operate the pump, the maximal 
uncertainty caused by imprecision in measurement of volume and time is now 
13.2(5 + e < 0.3; thirdly, the new objective function is 

^ 2avo+i.i{tl-tl+tl-tl+tl-tl-4:0ti+iat2~iot3+iati-iat5+iot6)~i32.2 
■9 ~ 20 

For this model, wc get the following results. 

— Using interval [5.2, 8.1], the optimal average accumulated oil volumcl ^^ = 
7.35 is obtained, which is a 7.5% improvement over the optimum 7.95 in [3]. 

— The local controllers for vq £ [5.2,8.1] is illustrated by I of Fig. [5j 



J-- — 13- 

_ io^o- 



13 
_10^>0-l-26 
, "-l" 13 



At2 = 
f\t2 = 
f\t2 = 
At2 = 



■ At3 



_ 5^0 + 76 



Do6[5.2,6.8) 

51.0+98 . . 51.0+92 „ , 20^0+3095 ^ r^ o 7 kN 

' — — At6 = — "9 At6 = 'xgg «o6l6.8,7.5) 



At4 



_ 51.0+98 



Ats 



_ 51.0+92 



Ate 



_ 51.0 + 110 



""I'a""" A t3 = li A t4 = ^ A t5=20 A t6=20 



«o6[7.5,7.8) 
«oe[7.8,8.1]. 



The local optimal value for wq G [5.2,8.1] is illustrated by II of Fig. [SJ from 
which it can be estimated that the mean average accumulated oil volume in 
the long run is around 6.8. 





Fig. 5. Optimal controllers and average accumulated oil volumes for 3 activations. 



^ The optimal value is first obtained at the 4th iteration, but there are many other 
intervals other than [5.2,8.1] that give the same optimal value 7.35. 



Furthermore, the following theorem indicates that the theoretically optimal 
controller can be obtained using the local control strategy with three activations. 
Therefore, our approach in fact gives the theoretically optimal controller in the 
oil pump industrial example. 

Theorem 1. For each admissible [L,U], each vq G [L,U], and any local control 
strategy S4 with at least 4 activations subject to Riu, Ri and Rig, there exists a 
local control strategy S3 subject to Riu, Ri and Ris with 3 activations such that 
20 Jt=o''^s3{t)dt < -^ Jt^QVs4^{t)dt, where Vs^it) (resp. Vs^{t)) is the oil volume 
in the accumulator at t with S3 (resp. S4). 

Proof. From the consumption rate of the machine in Fig. [5] and the behavior of 
the pump, we only need to consider a controller S4 that turns on the pump 4 
times in a circle in order to guarantee Riu , Ri and Ris . Furthermore, by Fig. [21 
it is easy to argue that turning on the pump can only take place in the intervals 
[2,4], [8,12] and [14,20] in order to obtain an optimal local control strategy; 
otherwise, a better local control strategy can be constructed just by postponing 
the activation time accordingly. In addition, we can further show that the pump 
can only be turned on at most once in the interval [8,12] for any optimal local 
control strategy. Now suppose we have an optimal local control strategy S4 that 
needs to turn on the pump four times in a circle in order to guarantee Riu, Ri 
and Ris- Then by the above analysis, S4 switches the pump on respectively in 
[2,4], [8,12], at 14 for 2 seconds and at 18 for another 2 seconds. If not, it is 
easy to show the strategy is not optimal by contradiction. Now, let us construct 
a local control strategy S3 that turns on the pump three times in a circle as 
follows: its first two activation time are the same as the counterparts of S4's, 
but last e seconds longer by considering noise, and it turns on the pump the 
third time at 14 for 3.2 + e seconds, where e is the noise (0.1 in this paper). 
By a simple calculation, it is easy to sec that S3 satisfies Riu, Ri and Ris, and 

M /t=o "^3 (i)di < Jo /t=o "^4 (i)di. □ 



7 Conclusions 

In this paper, we propose a "hybrid" approach for synthesizing optimal con- 
trollers of hybrid systems subject to safety requirements by first reducing the 
problem to QE and then combining symbolic computation and numerical com- 
putation for scalability. We illustrate our approach by a real industrial case of 
an oil pump provided by the HYDAC company. 

Compared to the related work, e.g. [3], our approach has the following ad- 
vantages. 

1. By modeling the system, safety requirements as well as optimality objectives 
uniformly and succinctly using first-order real arithmetic formulas, synthesis, 
verification and optimization are integrated into one elegant framework. The 
synthesized controllers arc guaranteed to be correct. 



2. By combining symbolic computation with numerical computation, we can 
obtain both high precision and efBciency. For the oil pump example, our 
approach can synthesize a better (up to 7.5% improvement of [3]) optimal 
controller in a reasonable amount of time (see Table [1]), even nearly a theo- 
retically optimal controller by Theorem [1] 

The issues of evaluation and implementation of our controllers are being 
considered. To make our approach more general with symbolic and numerical 
components, and apply it to more examples in practice will be our future work. 

Acknowledgements. Special thanks go to Mr. Quan Zhao for his kind help in 
writing an interface between different QE tools, and to Dr. David Monniaux for 
his instructions on the use of the tool Mjollnir. 
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A Display of Formulas by QE 

A.l The First 10 Disjuncts of I> 

26ii - lOuo - 157 > A 22ti - 22*2 + 22<3 - 22<4 - IOwq + 275 < A 

llti -11^2 + 11^3- llt4-5i;o + 5C/ + 65 > A llti - IU2 + IU3 - IU4 - 5va + 5L + 77 < OA 
fi-t2 + 2<0At2-i3 + 2<0At3-<4 + 2<0At4-20<0A2i;o-31>0Ai)o-C/<0A 
lOL - 51 > A IOC/ - 249 < 

V 26f 1 - lOwo - 157 > A 22<i - 22^2 + 22^3 - 22^4 - lOi'o + 275 > A 

22fi - 22^2 + 6^3 - lOuo + 95 < A ll<i - IK2 + 11^3 - 11^4 - 5wo + 5L + 77 < A 
fi-t2 + 2<0At2-i3 + 2<0At3-<4 + 2<0At4-20<0A2i;o-31>0A 
Wo - [/ < A IOC/ - 249 < A lOL - 51 > 

V 26fi - lOuo - 157 > A 22ti - 22^2 + 22^3 - 22^4 - lOi^o + 271 < A 

22ii ~ 22^2 + 18^3 - 10i)o - 97 > A llti - 11*2 + 11*3 - 11*4 - 5t;o + 5L + 77 < A 
ti - t2 + 2 < A t2 - *3 + 2 < A *4 - 20 < A 2z;o - 31 > A Wo - t^ < A 
IOC/ - 249 < A lOL - 51 > 

V 22fi - 11^2 - lOwo + 183 > A 22ii - 22^2 + 22^3 - 11^4 - IOijq + 183 > A 
22fi - 22^2 + 22^3 - 22t4 - IOwq + 341 < A 13*i - IOijq + 25 < A 

11*1 - 11*2 + 11*3 - 11*4 - 5i;o + 5C/ + 65 > A ll*i - 11*2 + 11*3 - 11*4 - 5wo + 5L + 77 < A 
*i-*2 + 2<0A*i-2>0A*2-*3 + 2<0A*3-*4 + 2<0A*4-20<0A 
Wo - C/ < A 1)0 - i > A IOC/ - 249 < A lOL - 51 > 

V 22fi - 6*2 - lOuo + 117 < A 22ti - 11*2 - lOwo + 183 > A 

22f 1 - 22*2 + 22*3 + 4*4 - lOfo - 157 > A 22*i - 22*2 + 22*3 - 22*4 - lO^o + 341 < A 
11*1 - 11*2 + 11*3 - 11*4 - 5uo + 5C/ + 65 > A 11*1 - 11*2 + 11*3 - 11*4 - 5t;o + 5L + 77 < A 
*i-*2 + 2<OA*i-2>OA*3-*4 + 2<OA*4-20<OAi'o-C/<OAi;o-i>OA 
IOC/ -249 < A lOL-51 > 

V 22*1 - 11*2 - lOwo + 183 > A 22*i - 22*2 + 22*3 - 11*4 - lOi^o + 183 > A 
22*1 - 22*2 + 22*3 - 22*4 - lOt^o + 341 < A 22*i - lOi^o + 73 < A 

11*1 -11*2 + 11*3- ll*4-5uo + 5C/ + 65 > A 11*1 - 11*2 + 11*3- ll*4-5wo + 5L + 77< OA 
*i-*2 + 2<0A*i-2>0A*4-20<0Ai;o-C/<0Ai;o--^>0A 
IOC/ -249 < A lOL-51 > 

V 22*1 - 11*2 - lOwo + 183 > A 22*i - 22*2 + 22*3 - 6*4 - lOwo + 117 > A 

22*1 - 22*2 + 22*3 - 11*4 - lOwo + 183 < A 22*i - 22*2 + 22*3 - 18*4 - lOi'o + 309 > A 
11*1 -11*2 + 11*3- ll*4-5i)o + 5C/ + 65 > A 11*1 - 11*2 + 11*3- ll*4-5wo + 5L + 77< OA 
*i-*2 + 2<0A*i-2>0A*2-*3 + 2<0Ai)o-C/<0Ai;o-i>0A 
IOC/ - 249 < A lOL - 51 > 

V 22*1 - 11*2 - lOwo + 183 > A 22*i - 22*2 + 22*3 - 22*4 - lOi^o + 341 > A 
22*1 - 22*2 - lOwo + 271< A ll*i - 11*2 + 11*3 - 11*4 - 5wo + 5C/ + 65 > A 
11*1 - 11*2 + 11*3 - 11*4 - 5i'o + 5L + 77<0A*i-*2 + 2<0A*2-*3 + 2<0A 
*3 - t4 + 2 < A *4 - 20 < A 2iJo -31>0A'yo-C/<0Auo--^>0A 

IOC/ - 249 < A lOL - 51 > 

V 22*1 - 11*2 - lO'yo + 183 > A 22*i - 22*2 + 22*3 - 6*4 - lOwo + 117 > A 

22*1 - 22*2 + 22*3 - 11*4 - lOi^o + 183 < A 22*i - 22*2 + 22*3 - 18*4 - lOi'o + 309 > A 

22*1 - 22*2 - lOwo + 271< A ll*i - 11*2 + 11*3 - 11*4 - 5i;o + 5C/ + 65 > A 

*i - 2 > A *2 - *3 + 2 < A 2wo - 31< A wo - L > A IOC/ - 249 < A lOL - 51 > 



V 22ti - 22^2 + 18^3 - lOi^o - 97 > A 22ti - 22^2 + 6^3 - lOwo + 95 < A 

22ii - lOuo - 109 < A 13ti - IOwq - 27 < A llti - lltz + lltg - llt4 - 5vo + 5t/ + 65 > A 
llti - llt2 + 11^3 - llt4 - 5i;o + 5L + 77 < A fi - t2 + 2 < A ti - 2 > A 
i3 - t4 + 2 < A t4 - 20 < A lOwo -77>0Awo-f/<0Ai;o-i>0A 
IOC/ - 249 < A lOL - 51 > 

A.2 The First 10 Disjuncts of !>' 

ii - 14 = A f2 - 16 = A f3 - 18 = A i4 - 20 = A 

lOwo - 207 = A lot/ - 207 > A IOC/ - 249 < A lOL - 141 = 

V fi - 14 = A f2 - 16 = A f3 - 18 = A i4 - 20 = A 

lOuo - 187 = A IOC/ - 187 > A IOC/ - 249 < A lOL -121 = 

V fi - 14 = A f2 - 16 = A f3 - 18 = A i4 - 20 = A 

lOwo - 187 > A 10i;o - 207 < A 5wo - 5L - 33 = A i^o - C/ < A IOC/ - 249 < 

V 22f 1 - lOuo -121 = 0A<i-<2 + 2 = 0Ati-t3+4 = 0At4-20 = 0A 

llfi - 138 > A ti - 14 < A 22fi - IOC/ - 121 < A lOC^ - 249 < A lOL - 121 = 

V llfi - 5uo + 5L - 121 = A ti - t2 + 2 = A fi - ^3 + 4 = A <4 - 20 = A 

f 1 - 14 < A 2uo - 31 > A Wo - C/ < A IOC/ - 249 < A 26<i - lOi^o - 157 > A 
22fi - lOwo - 121< A IKi - 5wo + 5C/ - 133 > 

V f 1 - 14 = A f2 - 16 = A f3 - 18 = A f4 - 20 = A 

lOwo - 187 = A IOC/ - 187 > A IOC/ - 249 < A lOL - 51 > A lOL - 121< 

V 22ii - lOuo -121 = 0Ati-t2 + 2 = 0Afi-f3+4 = 0At4-20 = 0A 
IOC/ - 249 < A lOL - 51 > A lOL - 121 < 022ii - IOC/ - 121 < A 
Allti - 138>0 A fi -14< 

V 26f 1 - lOuo - 157 = A 4ti - lOL + 85 = A ti - f2 + 2 = A f 1 - t3 + 4 = A t4 - 20 = A 
ii - 12 > A fi - 14 < A IOC/ - 249 < A 26ti - lOC^ - 157 < A 4ti - IOC/ + 109 < 

V f 1 - 14 = A f2 - 16 = A ^3 - 18 = A i4 - 20 = A 

lOwo - 207 = A IOC/ - 207 > A IOC/ - 249 < A lOL - 51 > A lOL - 141< 

V llfi - 5uo + 5C/ - 133 = A llti - 5uo + 5L - 121 = A ti - ^2 + 2 = A ti - ^3 + 4 = A 
i4 - 20 = A 2uo - 31 > A 26fi - lOi^n - 157 > A llti - 133 < 



